Lexical based two-way RTE System at RTE-5

The note describes the lexical based two-way Recognizing Textual Entailment (RTE) system developed at the Computer Science and Engineering Department, Jadavpur University, India. We participated in the two-way main task at RTE-5. The system is based on the composition of the following six lexical based RTE methods: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and named entity matching. Each of these methods were applied on the development data to obtain two-way decisions. It was observed on the development data that final entailment decision on a text-hypothesis pair that is based on positive entailment decisions from at least two lexical based RTE methods was producing a better precision and recall figure. An accuracy figure of 58.17% was obtained on the test data. Ablation tests were performed for each of the six RTE methods and these are reported in the present note. The RTE task was based on three application settings: QA, IE and IR but this information was not taken into consideration during the system development. The relatively higher accuracy figures for the IR application setting obtained during the various tests suggest that identification of appropriate RTE methods based on the application settings might have improved the accuracy scores further.